Async mmap population by olpipi · Pull Request #36668 · openvinotoolkit/openvino

olpipi · 2026-07-01T14:11:48Z

Details:

item1
...

Tickets:

ticket-id

AI Assistance:

AI assistance used: no / yes
If yes, summarize how AI was used and what human validation was performed (build/tests/manual checks).

praasz · 2026-07-03T14:03:02Z

+ * 0).
 */
-void vm_prefetch(void* ptr, size_t size, size_t num_threads = 0) noexcept;
+PrefetchToken vm_prefetch_async(void* ptr, size_t size) noexcept;


Old function should be preserved.

vm_prefetch_async with the same input parameters.
Is possible to hide the token from user (use void) and fire the fetch in background?

praasz · 2026-07-03T14:03:45Z

+     *               mapping when set to auto_size.
+     * @return A @ref util::PrefetchToken owning any spawned worker threads (empty by default).
+     */
+    virtual util::PrefetchToken hint_prefetch_async(size_t offset = 0, size_t size = auto_size) {


Skip it for now lets focus on util that will work then we decide how it work.
If kept make pure virtual

praasz · 2026-07-06T06:12:39Z

 }

-void vm_prefetch(void* ptr, size_t size, size_t num_threads) noexcept {
+void vm_prefetch(void* ptr, size_t size, bool fast) noexcept {


Dont change it

praasz · 2026-07-06T06:15:47Z

+    }
+
+    void hint_prefetch_async(size_t offset, size_t size) override {
+        if (const auto plan = util::make_prefetch_plan(m_data, m_size, offset, size); plan.m_aligned_size) {


This change is not required old function can be use just set caculated region into new fetch function

praasz · 2026-07-06T06:17:31Z

+     * Starts populating the given region of the mapping in background threads and returns
+     * immediately. Unlike the lower-level @ref util::vm_prefetch_async, no token is handed back
+     * to the caller: the implementation is fully responsible for tracking the background work
+     * internally and joining it before this object is destroyed (so the mapping is never torn
+     * down while a background task might still be touching it), regardless of what the caller
+     * does afterwards. This avoids requiring the caller to correctly manage a token's lifetime
+     * relative to this object's lifetime, which is an easy way to introduce a use-after-free.


The comment should be updated (make it shorter) information of token etc is not required. It should give information that start fetch data in async way.

almilosz · 2026-07-07T12:29:13Z

 #include <sstream>
+#include <thread>
+#include <utility>



I can see that you added a new "compute" scenario test to FileLoadBenchmark pull/36741
Do you plan to extend FileLoadBenchmark with hint_prefetech_async in this pr?

praasz · 2026-07-16T07:50:57Z

+ * @brief Number of parallel chunks a @p size byte job should be split into: at least one, at most
+ * @p max_chunks, targeting roughly one chunk per @p min_chunk bytes.
+ */
+constexpr size_t split_chunk_count(size_t size, size_t min_chunk, size_t max_chunks) noexcept {


Lets keep as internal util function.

praasz · 2026-07-16T07:51:56Z

 }

 void vm_decommit(void* ptr, size_t size) noexcept {
-    assert(ptr != nullptr && size > 0);


Why removed?

praasz · 2026-07-16T07:52:11Z

 }

 void vm_release(void* ptr, size_t size) noexcept {
-    assert(ptr != nullptr && size > 0);


Why asserts are removed?

praasz · 2026-07-16T07:52:35Z

-    } else {
-        // blocks until every page has been faulted in.
-        populate_pages(ptr, size, num_threads);
+        return;


Keep if else block instead

praasz · 2026-07-16T07:52:41Z

 }

 void vm_prefetch(void* ptr, size_t size, size_t num_threads) noexcept {
-    assert(ptr != nullptr && size > 0);


Why removed?

praasz · 2026-07-16T07:53:18Z


    void hint_evict(size_t offset, size_t size) noexcept override {
        if (m_mapped_view != MAP_FAILED) {
-            if (const auto region = util::make_madvise_region(m_data, m_size, offset, size); region.m_length > 0) {


Why changed? this is not part of this PR

praasz · 2026-07-16T07:53:36Z

- * @param size         The size of the region.
- * @return AlignedRegion The aligned memory region.
- */
-inline util::AlignedRegion make_madvise_region(const void* data, size_t mapping_size, size_t offset, size_t size) {


This is not part of this PR

praasz · 2026-07-16T07:54:18Z


    void hint_prefetch(size_t offset, size_t size) override;

+    void hint_prefetch_async(size_t /*offset*/, size_t /*size*/) override {}


Why there is no windows implementration?

praasz · 2026-07-16T07:57:03Z

 endif()

 target_include_directories(${TARGET_NAME} PUBLIC $<BUILD_INTERFACE:${UTIL_INCLUDE_DIR}>)
+target_include_directories(${TARGET_NAME} PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src)


Detail:
Just expand line above with PRIVATE section instead add target_include_directories new entry

praasz · 2026-07-16T07:58:02Z

    const size_t hw_threads = (std::max)(size_t{1}, static_cast<size_t>(std::thread::hardware_concurrency()));
-    const size_t max_by_size = size / (1024 * 1024);  // 1 thread per MB
-    const size_t num_threads = (std::max)(size_t{1}, (std::min)(hw_threads, max_by_size));
+    const size_t num_threads = split_chunk_count(size, one_mib, hw_threads);  // 1 thread per MB, capped by HW


Lets not touch this source focus on mmap population only

github-actions Bot added the category: Core OpenVINO Core (aka ngraph) label Jul 1, 2026

olpipi added do not merge do_not_review do_not_merge do-not-merge Do not merge this PR and removed category: Core OpenVINO Core (aka ngraph) labels Jul 1, 2026

github-actions Bot added the category: Core OpenVINO Core (aka ngraph) label Jul 1, 2026

olpipi marked this pull request as ready for review July 2, 2026 15:35

olpipi requested review from a team as code owners July 2, 2026 15:35

olpipi removed the do_not_review label Jul 2, 2026

almilosz mentioned this pull request Jul 3, 2026

[Core] Add parallel mapping population on Windows #36586

Merged

olpipi force-pushed the async_mmap_population branch from 387c908 to ec1fca8 Compare July 3, 2026 16:38

olpipi removed do not merge do_not_merge do-not-merge Do not merge this PR labels Jul 3, 2026

praasz reviewed Jul 6, 2026

View reviewed changes

almilosz self-assigned this Jul 7, 2026

almilosz reviewed Jul 7, 2026

View reviewed changes

olpipi force-pushed the async_mmap_population branch from ec1fca8 to 06f279d Compare July 15, 2026 12:34

olpipi requested review from a team as code owners July 15, 2026 12:34

github-actions Bot added category: IE Tests OpenVINO Test: plugins and common category: build OpenVINO cmake script / infra labels Jul 15, 2026

olpipi force-pushed the async_mmap_population branch 3 times, most recently from 38a3431 to 5677bb3 Compare July 15, 2026 12:40

praasz added this to the 2026.4 milestone Jul 15, 2026

olpipi added 2 commits July 15, 2026 16:58

Async mmap population on Linux

e96e0db

Refactor and comments

f733690

fixes

596d001

olpipi force-pushed the async_mmap_population branch from 6c189f2 to 29d58bc Compare July 15, 2026 15:07

fix

ab5e330

olpipi force-pushed the async_mmap_population branch from 29d58bc to ab5e330 Compare July 15, 2026 15:08

Fix clang-format

9a894ff

praasz reviewed Jul 16, 2026

View reviewed changes

praasz assigned praasz and t-jankowski Jul 16, 2026


		void hint_prefetch(size_t offset, size_t size) override;

		void hint_prefetch_async(size_t /offset/, size_t /size/) override {}

Uh oh!

Conversation

olpipi commented Jul 1, 2026

Details:

Tickets:

AI Assistance:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants